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1. Introduction 

Our goal in this paper is to prove a formula for the general degeneracy locus Q r 
associated to an oriented quiver of type A n . If we are given a sequence of vector 
bundles and vector bundle maps 

TP jtK TP jt\ TP j. TP tHu TP 

on an algebraic variety X, and a collection r = ('"i J )o<i<j<n °f non-negative inte- 
gers, there is a degeneracy locus fi r = Q r (E.) = Q r (E.,(j>.) defined by 

(1.1) Q r = {xeX\ rank(Ei(x) -> Ej(x)) < r y V i < j} . 

This is a closed subscheme of X; locally, where the bundles are trivial, this is defined 
by vanishing of the minors of size nj + 1 in the product of matrices giving the map 
(fij o • • • o from Ei to Ej, for all i < j. 

Not all rank conditions give reasonable loci. Those that do — and the only ones 
we will consider — are characterized by the conditions 

fij < r i.j-i an d Tij < r ll+ i,j for all i < j, and 
n+ij-i - - n+i j + r tj > for alH < j — I, 



where we set ru — rank(£"i). In fact, rank conditions satisfying (1.2) are the only 



conditions that can actually occur, i.e. for which one can have equality in (1.1) 



When the maps are sufficiently generic, each such f2 r is irreducible, of codimension 

(1.3) d(r) = ^2(r it j-i - rij)(r i+ld - r y ) . 

i<j 

When n = 1, the formula for O r is the well-known Giambelli-Thom-Porteous 
formula, which we recall in order to introduce some notation. For a map <f) : E — > F 
of vector bundles of ranks e and /, and a non-negative integer r < min(e, /), Q r is 
the locus where <j) has rank at most r. The formula for f2 r is the Schur polynomial 

S (e _ r) f~r(F - E) , 

which is defined as follows. Define cohomology classes hi by the formula hi — 

c(E v )/c{F v ), where c(E v ) = 1 - c x (E) + c 2 (E) is the total Chern class, and 

the division is carried out formally; in particular, ho — 1 and hi — for i < 0. For 
any sequence A = (Ai, . . . , A p ) of non-negative integers, set 

s\{F — E) = det(/i Ai+J _ i )i< !i)J <p . 

In the Giambelli-Thom-Porteous formula, A — (e — r)f~ r denotes the sequence 
e — r repeated / — r times. In a Schur determinant s\(F — E), A will usually be a 
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partition, i.e. a weakly decreasing sequence, but later we will also need this notation 
when A is not a partition. 

Our general formula for the locus O r , when r is any set of rank conditions 



satisfying (1.2), has the form 

c\(r) s x (E.). 



A 



where the sum is over sequences A = (A(l), A(2), . . . , A(n)), with each X(i) a parti- 
tion. The class s\(E.) is defined to be 

S\{E.) = 8\(ty(El - Eq) ■ Sx(2)(E 2 - Ei) ■ . . . ■ Sx( n ){E n - E n -l) . 

The coefficients c\(r) are certain integers for which we give an inductive formula. 

A second purpose of this paper is to introduce these integers c\(r), which we 
regard as generalized Littlewood-Richardson coefficients. We have a conjectured 
formula for c\(r) as the number of sequences (Ti,T2, ...,T n ) of Young tableaux, 
with Ti of shape X(i), satisfying certain conditions. This formula has been proved 
when the number of bundles is at most four, but it appears to be a difficult com- 
binatorial problem to prove it in general. 

In Q a special case of this situation was studied, where the rank conditions are 
given by a permutation w. For maps 

G\ — > G2 — > • • • — > G m — > F rn — * F m _i —¥■•'—¥ Fi 

with rank(Gi) = rank(_Fi) = i, and w G S m +i, let 

Cl w = {x G X I rank(G g (x) — > F p (x)) < r w (p, g) V p, 3 < m} , 

where r w (p,q) = #{i < p | w(i) < q}. These loci are special cases of the loci Q r 
described in this paper. The formulas given here therefore specialize to the universal 
double Schubert polynomials & w (c.(F.); c. (G.)) for these loci. Since these universal 
Schubert polynomials specialize to quantum and double Schubert polynomials (0, 
HUi we derive formulas for these important polynomials. These formulas 
appear to be new even for the single Schubert polynomials & w (x). 

Among the loci considered here are the varieties of complexes, which are the loci 
fl r with r,j = for j — i > 2. In this case the formula for the coefficients c\(r) is 
particularly simple, and it agrees with our general conjectured formula. 

In Section |^ we discuss the loci Q r in more detail, state the main theorem, and 
derive the main applications. This includes a precise statement of what it means 
for a polynomial ^2 c\(r)s\(E,) to give a formula for a locus fl r . This statement 
implies the assertion that if X is non-singular and Q r has the expected codimension 
d(r), then 

(1.4) p r ]=Yc x (r)s x (E.) 



in the Chow group A d<J, \X). However, weaker assertions can be made when X 
is singular or the maps fa are less generic. At the end of Section || we sketch a 
generalization, which is based on explicit resolutions of singularities of these loci. 

The coefficients c\ (r) are determined by the geometry, if this assertion is inter- 
preted correctly. We will see in Section || that c\ (r) depends only on the differences 
r;.j_i — r,j and rj+ij — r#. This allows the ranks of the bundles Ei to be taken large 
compared to the expected codimension d(r); if the Chern classes of the bundles are 
independent, the coefficients c\(r) are then uniquely determined by ([L 
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Much of the work in a project of this kind — discovering the shape of the formula — 
is invisible in the final product, which has a short proof (given in Section |^). In 
particular, it came as a pleasant surprise to us that the polynomials for all the loci 
f2 r can be written as a linear combination of the polynomials s\(E.). We know of 
no reason for this other than the proof of the explicit formula. That the coefficients 
c\(r) appear to be non-negative is even more surprising. 

The conjectured formula for the coefficients is discussed in more detail in the 
final Section ||; proofs of the combinatorial assertions made there can be found in 

@ 

We are particularly grateful to S. Fomin, who provided an involution on pairs of 
tableaux which gave us the strongest evidence for the conjectured formula, and who 
has collaborated with us on the combinatorial aspects of this problem. Thanks also 
to M. Haiman and M. Shimozono for their responses to combinatorial questions. 
The Schubert package jl2| was useful for calculations. 



2. Quiver varieties; the theorem and applications 

2.1. The Main Theorem. Given vector bundles Eq, . . . , E n on a variety A, let 
H be the direct sum of the bundles Hom(£i_i, Ei), i.e. 

H = Hom(E ,Ei) x x Rom(Ei,E 2 ) Xx • ■• x x Hom(J5 n _i,£7 n ) . 

Writing Ei for the pullback of Ei to H, we have a universal or tautological sequence 
of bundle maps 

(2.1) E — U Ei — ^> E 2 — > • • • — » E n -\ — ^* E n 

on H. For this universal case, it is a theorem of Lakshmibai and Magyar |Q that 
for r satisfying (1.2), the scheme fl r — fl r (E.) for ( |2.l| ) is reduced and irreducible, 



of codimension d(r). Moreover, Cl r is a Cohen-Macaulay variety if A is Cohen- 
Macaulay. (Earlier Abeasis, del Fra, and Kraft [|| had shown, in characteristic 
zero, that the reduced scheme (fi r ) re d is Cohen-Macaulay.) Note that, when the 
bundles are trivial, H is a Cartesian product of X and a product M of spaces of 
matrices, and Q r is a product of X with the corresponding locus in M\ it is this 
locus in M that is studied in (l), |§, and @. 

The statement that "the polynomial P = ^2 c\(r)s\(E,) is a formula for the 
locus r2 r " has the usual meaning in intersection theory (cf. §14], [JlO], App. A]). 
It implies that when X is non-singular and codim(r2 r , X) — d{r), then 

PA =$> A (r) SA (£.) 

A 

in the Chow group A d ^X, where [Q r ] is the cycle defined by the scheme fl r . 
For arbitrary X and maps <pi, there is a well defined cycle class tt r in the Chow 
group A m _M r )(Q r ) , where m = dim(X), whose image in A m _ d r r \(X) is the class 
c *{ r ) s \{E.) n [A]. Whenever O r has codimension d{r) in A, fl r is a positive cycle 
supported on fi r ; if A is Cohen-Macaulay, or more generally if depth(f2 r , A) = d(r), 
this cycle is [fi r ] , but if A is not Cohen-Macaulay the coefficient of a component of 
fi r in fl r may be smaller than the length of fl r at its generic point. These classes 
fl r are compatible with the basic constructions of intersection theory, exactly as in 
§ Thm. 14.3]. 
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In fact, to give maps <pi : — > -Bj for all i is the same as giving a section 

s : X H of the bundle H, and £! r = s -1 (f2 r ). The general class Q r is constructed 
by intersecting fl r C H with the (regular) embedding s : X — > iJ, i.e. 

n r = s ! [f2 r ] , 

where s ! : A*(f2 r ) — > A*(0 r ) is the refined intersection |6|, §6]. As in |(| §14], the 
general properties of these classes follow from this construction. It therefore suffices 
to prove the corresponding formula on H , i.e. that 

fir}=J2cx(r)s x (E.)n[H} 

X 

in j4jv-d(r)(-ff)) where N = dim(if) = dim(X) + J2i=i e i-i e i, e i = rank(Sj). 
It is natural to arrange the rank conditions in a triangular array: 



roo ru r 22 r 33 

rot r u r 23 

n>2 ri 3 

^03 ' ■ ' r n -3,; 



ror, 



It is useful to replace each small triangle 



e / 
r 



occurring in this array by the rectangle of width e — r and height f —r. 

e — r 



f-r 



We then have the rectangular array 

i?oi R12 R23 

R02 Rl3 

Ro3 



R 



n—3,n 



Rn- 



l.n 



Rn 



R 



On 

where Rij has width rij-i — and height rj+ij — r^. Note that the expected 
codimension d(r) is the sum of the areas of the rectangles. The condition (1.2) 
says that the rectangles get (weakly) shorter as one proceeds in a southeasterly 
direction, and they get (weakly) narrower as one travels southwest. For example, 
the rank conditions given in the triangular array 

6 8 9 6 
5 6 6 
4 3 
2 
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correspond to the rectangular array: 



B 



m 



Our formula depends on the rectangles in this array. To be precise, it depends 
on the integers Tij-\ — r^j and r<+ij — r^j for all i < j\ if a width r^j-i — rij 
is zero, we need to know the height r»+ij — rij, even though the rectangle Rij is 
empty. (The conjectured formula discussed later does not have this defect.) Each 
Rij is identified with the partition — nj) ri+1 > j ~ ni for which it is the Young 

diagram. 

At this point we need some notation. If R is a rectangle of width e and height 
/, and <7 and r are partitions, with the length £(a) of a at most /, then (R + a, r) 
denotes the sequence (e + ci, e + 02, . . . , e + <x/, Ti,T2, . . . ); this is a partition only if 
e + (7f > T\. For a partition A = (Ai, . . . , A p ), |A| denotes ^ Ai, which is the number 
of boxes in the Young diagram of A. For partitions A, c, r with \a\ + |r| = |A|, r 
denotes the Littlewood-Richardson coefficient, which is the coefficient of the Schur 
polynomial s\ in the expansion of s a ■ s T (see |l7j). We set = ra = rank(i?i), 
r,i = ri—ii, and Ri = Ri-i t i, so Ri has height — and width e^-i — r,. 

If / = (ii, . . . , i p ) is a sequence of non-negative integers that is not weakly 
decreasing, then si(F-E) is either or it is is\(F — E) for some unique partition A 
and unique coefficient ±1. This partition and coefficient can be found by performing 
a sequence of moves of the type 



if jk+i > jk', if one reaches a sequence (J±, . . . ,j p ) with some jk+i = jk + 1, then 
Si = 0; otherwise one reaches a partition A = (Ai, . . . , A p ) in m steps, and then 



We now give an algorithm for constructing finite formal sums ^2 c a(^)5'(A), with 
A varying over n-tuples of partitions A = (A(l), . . . , A(n)). The polynomial for the 
degeneracy locus f2 r will be obtained by replacing each S(X) by s\(E.). In the 
algorithm we will meet symbols S(I(1), . . . ,I(n)) where each is a sequence 
of integers. For such symbols we imitate the above rule for Schur polynomials 
to write S(I(l), . . . , 1(1)) as either zero or ±5(A(1), . . . , A(n)) for unique parti- 
tions A(l), . . . , X(n). If s/y) = for any j, put S(I(1), . . . , /(ri)) = 0; otherwise 
write Sj(j) = ejSA(j) f° r 1 < j < n, with ej = ±1, and put S(I(1), . . . , I(n)) — 



We construct the polynomial c\(r)S(X) by induction on n. For n = 1 we have 
just one rectangle R = Rqi, and the polynomial is S(R), which gives sr(Ei — Eq). 
Given the rectangular array for r, delete the top row. This gives a smaller array, 
for which we have a polynomial ^d^S^) by induction, the sum over sequences 
/i = (/i(l ), . . . , [i(n — 1)) of partitions. The polynomial c\(r)S(X) is obtained by 
replacing each S(fi) in J^d^Sifi) by 



(ji, ■■ -,j P ) | -» (jli ■ • ■ , jk-l, jk+i - l,jfc + 1, jk+2, ■ ■ ■ ,jp) 



Sl = {-l) m S X . 



(Il^)S(A(I),...,A(n)). 
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Here the sum is over all sequences (c(l), . . . , a(n — 1)) and (r(l), . . . , r(n — 1)) 
of partitions, with \a(i)\ + \r(i)\ = |ju(i)|, such that the length of a(i) is at most 
the height of Ri, i.e. t(cr(i)) < &i — r.;. Define to be (Ri + a(i),r(i — 1)) for 
1 < i < n, where r(0) and <r(n) are taken to be the empty partition. One uses 
the rules just given to write each S(I(1), . . . , I(n)) as or ±S'(A) for a unique 
A = (A(l), . . . ,X(n)), thus arriving at a polynomial c \( r )S(X)- 

Main Theorem. The formula for fl r is c \( r ) s \(E.). 

This theorem will be proved in the next section. We first interpret it in the 
case where the rectangular array has only two non-empty rows, i.e. R^ — if 
j — i > 2. In this case the inductive polynomial ^2d fi S(fi) is just S(fi), for 
/i = (Rq2, R13, ■ ■ ■ , R n -2,n)- For a rectangle shape R, the Littlewood-Richardson 
coefficient c^ r vanishes unless a and the 180° rotation of r fit together to make R, 
in which case c^ T = 1. 





a 1 


R = 





Corollary 1. // Rij is empty for j — i > 2, then the formula for fl r is ^2 s\(E.), 
where the sum is over all A = (A(l), . . . ,X(n)), with = (Ri + o~(i),r(i — 1)), 
such that o~(i) and r(i) fit together to form Ri-\^+\ for 1 < i < n — 1; here cr(n) 
and r(0) are empty. 



Note that, by (1.2), for any division of into a(i) and r(i), o~(i) always 

fits on the right side of Ri, and t(i) fits below Ri+i, so the resulting sequences 
X(i) = (Ri + <y(i), r(i — 1)) are always partitions. This formula can be remembered 
by the picture 





The situation in the corollary covers the case of varieties of complexes, which 
means that r^ — for j — i>2. In this case the triangular array is 



eo ei e 2 e?, ■ ■ ■ e n 

ri r 2 r 3 ■■■ r n 
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so the array of rectangles is 

e - r 1 ei 



&2 - r 3 





e-2 - r 2 




es - r 3 









e n -i 



n 



'2 



T3 



P. Pragacz reports that he had known this formula for il r in the case of varieties 
of complexes. 

2.2. Geometric description of c\(r). Although we use the notation c\(r) for 
the coefficients, it should be emphasized that they depend only on the differences 
rij—i — ry and fi+ij — fy, not on the integers themselves. For example they 
are unchanged if the same positive integer is added to each r*y . 

The linear independence of ordinary Schur polynomials s\(x\, . . . , x p ) as A varies 
over partitions of length at most p, implies that the polynomials s\(E.) are linearly 
independent functions of the Chern classes of the bundles Eq , E\ , . . . , E n , if the 
ranks of the bundles are suitably large (e.g. if £(X(i)) < ej for 1 < i < n). 

From the preceding two paragraphs it follows that the coefficients c\(r) are 
uniquely determined by the geometry, i.e. by the fact that ^2c\(r)s\{E.) is a 
formula for Q, r . To see this, one can choose the ranks large, and one can find a 
smooth variety X on which f2 r has the expected codimension d(r), and for which 
the classes s\(E,), for ~ d(r), are linearly independent. For example, one 

can start with universal bundles Ei on large Grassmanians Gi, let G = Wa—qG^ 
and set X = Hom(Ej_i, Ei). 

2.3. Schubert polynomials. In Jl|, the rank conditions r satisfying ( |l.2| ) are 
described by diagrams of dots connected by lines. One puts = ru dots in column 
i, and lines are drawn between some dots in adjacent columns. Then is the 
number of lines connecting a dot in column i to a dot in column j. The example 
given earlier in this section can be described by the diagram 



Now fix a positive integer m. For a permutation w G S m +x, we form a diagram 
with 2m columns of lengths 1, 2, . . . , to, to, m — 1, . . . , 2, 1. All possible lines are 
drawn among the first to columns and among the last to columns. Between the two 
middle columns, the i'th dot on the right is connected to the w(i)'th dot on the 
left. If w(i) = to + I, no connection is made. For example, if to = 4 and w = 31452, 
this diagram is 
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The number of connections between the left column with q dots and the right 
column with p dots is the number 

r w (p,q) = #{i < P I w{i) < q} . 

There are the maximal number of connections between two columns on the left or 
between two on the right. This means that for a sequence E, of bundle maps 

Gi — ► G-x — ► ■ ■ ■ — » G m — » F m — > F m _i i*i 

with rank(Gi) = rank(i 7 i) = i, the locus Q, r (E,) defined in the introduction is 
exactly the locus fl w defined in with the same scheme structure. In || "universal 
Schubert polynomials" & w (c.(F.);c.(G.)) were constructed, which represent the 
loci Q w . From the fact that the formula for a locus is unique, we deduce the 
following corollary. 

Corollary 2. With r determined by w as above, 

e w (c.(F.);c.(G.)) =^c A (r) s x (E.) . 

A 

When these bundle maps are specialized so that each G^_i — » G; is an inclusion 
of bundles, and each Fi — * is a surjection, then 6,i,(c.(F.); c.(G.)) becomes 
the double Schubert polynomial 

&w (^1 ) • • • 5 *£m ; yi i ■ ■ • 7 

of Lascoux and Schiitzenberger; here we set xi = ci(ker(_Fi — > -Fi-i)) and ?/i = 
Ci(Gi/G;_i). The right side of the formula in this corollary also simplifies in this 
case. It follows from the definition that for a partition r, we have 




if T= (q), q>0 
otherwise 



s r(Gi — Gi-i) — 

and 

ST (F i _ 1 -F i )=[ { - Xi)V if ^W P 'P^° 
[ otherwise. 

Thus s\{E.) = unless A = {{q 2 ), (93), ■ ■ ■ , {q m ),i~, (l) Pm , ■ ■ ■ , (1) P2 ), in which case 
s x (E.) = • --x^yf ■ --y^sAx/y) , 

where s T {x/y) — det(/i Ti+J _i), ^ hk = Yi(^ ~Vi)/ II (1 ~ x j)- Our formula therefore 
writes & w {x; y) as a signed sum of monomials in the X2, ■ ■ ■ , x m , y 2 , ■ ■ ■ , y m times 
Schur polynomials s T (x/y). When all variables yi are set equal to zero, we have 
only the terms with qi = • ■ • = q m = 0, and this writes the ordinary Schubert 
polynomial & w (x) — & w (x;0) as a signed sum of monomials in x 2 , ■ ■ ■ ,x m times 
(symmetric) Schur polynomials s T (x). 

Unlike the inductive construction of Schubert polynomials from high degree to 
low degree, our formulas are simplest for those of low degree. 
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For example, for w = 3142 G £4, the corresponding array of rectangles is 

[ □ 



□ 



□ 



Calculating ^2c\(r)S(X) with the algorithm of the main theorem, working from 
the bottom up, one has, with the empty partition 

5(0) ~» 5(0,1) 

~» 5(0, 1,0) + 5(0, 0,1) 
~» 5(0, 2, 0, 0) + 5(0, 1, 1, 0) + 5(0, 1, 0, 1) 
^ £c A (r)5(A) = 5(0, 2, 1, 0, 0) + 5(0, 1, 1 1, 0, 0) + 
5(0, 1, 2, 0, 0) + 5(0, 1, 1, 1, 0) + 5(0, 0, 2 1, 0, 0) + 
5(0, 0, 1 1, 1, 0) + 5(0, 1, 1, 0, 1) + 5(0, 0, 1 1, 0, 1) . 

Substituting s\(E.) for 5(A), this is a formula for the universal double Schubert 
polynomial associated to w — 3142. It specializes to the formula 

©3142(2:; y) = s 21 (x/y) + y 3 s 2 (x/y) + 

(2/3 -x 2 - x 3 )s n (x/y) + y 3 (y 3 - x 2 - x 3 )sx(x/y) 

and to 

63142(2;) = s 2 i(x) - (x 2 + x 3 )sn(x) . 

The rectangular array coming from a permutation w € 5 m _|_i has only empty 
rectangles and lxl rectangles. In fact, this array is determined from the diagram 
of the permutation denoted D'{w) in js|, §2]: the diagram D'(w) is reflected in a 
vertical line, then rotated 135° clockwise, and the result fitted in the bottom of the 
triangle; each box in D'(w) is then in the position of a non-empty rectangle of the 
rectangular array. In other words the rectangle R^j is non-empty iff D (w) contains 
a box in position (2m — j, i + 1), which happens exactly when w(2m + 1 — j) < i + 1 
and ui _1 (i + 2) < 2m — j. 

3. Proof of the Theorem 

3.1. Geometric preliminaries. It follows from the general discussion in Sec- 
tion |U that it suffices to prove the formula for the universal locus $l r (E.) in H = 
Hom(i? ! _ i,Ei). (Throughout this section we omit notation for pullbacks of bun- 
dles by canonical maps.) In particular, we know from Q and (l4) that il r (E,) is 
reduced and irreducible of the expected codimension d(r), and that £l r (E,) is the 
closure of the locus f2° (E. ) where each of the maps Ei (x) — > Ej (x) has rank equal 
to rij for i < j. We must prove that, with these assumptions, 

[n r (E.)}=J2cx{r)s x (E.)n[H] 
x 

in the Chow group A N _ d ( r - ) (H), N = dim(_ff). 



Form the Grassmannian bundle Go over X with r-i = fj-i,, as in §2.1 
G = Gr(n,-Ei) x x Gv(r 2 ,E 2 ) x x ■■■ Xx Gr(r n ,E n ) . 
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Let G = G x x H, with projection it : G — > H. Let —> A l — > E l — > Qi — » 
be the universal exact sequences on Go, and hence also on G. Let Z C G be the 
intersection of the zero-schemes of the canonical maps E^i — > — > Qi, i.e. 

z = z(£ -» Qi) n -> q 2 ) n • • • n z{E n - X -» Q„) . 

On Z we have maps — > A, for 1 < i < n. Composing these with the inclusions 
Ai-i C Ei-i we get a sequence A. of bundles and bundle maps on Z: 

A 1 -> A 2 -> >A n . 

Let f denote the rank conditions obtained by omitting the top row of the trian- 
gular array for r, and let ft? (A.) C Z be the locus given by these maps and rank 
conditions. It is easy to see that Sl?(A.) is mapped into Sl r (E.) by n. 

Now Z is isomorphic to the bundle Hom(i^_i, Ai) over Go, and we have 

a canonical projection 

n n 

p:Z = ($ Hom{Ei- U Ai) — > Hom(^_i, A») = if' . 

i=l i=2 

Denote by O' the universal locus ft? (A.) of H' . Then Of (A.) in Z is the inverse 
image of SI' by p. Since the maps on H' are universal, it follows that SI' is irreducible, 
and therefore Of (-4.) is an irreducible subscheme of Z. 

si' « — o f (A.) n r (s.) 

" 

H' — ^ >■ G i? 



G 

Lemma 1. tt maps Clf(A.) birationally onto Sl r (E.). 

Proof. Let Z° denote the open subset of Z where the maps £?,_i — > Aj are sur- 
jective. Then the schemes Sl^(A.) n Z° and Q°(i?.) are isomorphic; they are both 
universal objects in the category of schemes Y over H, such that the pullback to 
Y of the sequence E. satisfies rank(_Ei — > Ej) = m everywhere on Y. Since both 
Qr(A.) and Sl r (E.) are irreducible schemes, and since Sl°(E.) ^ 0, the assertion 
follows. □ 

Note that this argument gives a direct proof that the codimension of Sl r (E.) 
in H is d(r). Indeed, by induction we know that SI' has codimension d(f) in H' , 
and Qf{A.) must then have the same codimension in Z. We conclude that the 
codimension of Sl r (E,) in H is 

d(f) + dim(ff) - dim(Z) = d(r) . 

3.2. Proof of the main theorem. By induction on n we know that [SI'] = 
£c„(f)s„(A.)n 

= = £ c„(f) ^(A.) n [Z] . 
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Furthermore [Z] = Y\" =1 ct op (Hom(-Bj_i, Qi)) n [G], so 

n 

[Z} = Y[s R ,(Q l -E^ 1 )n[G}, 

i=i 

where R[ — (ei-i) ei ~ ri . Since 7r* [f2 F (^4.)] = |T2 r (-E.)] by Lemma we are therefore 
reduced to proving the identity 

(3.1) 7T* l^c^f)a^A.)-f[a^(Qi-Ek-i)n[(?i \ = ^ c A (r)s A (£.) n [#] ■ 

\ (i i=l / A 

For this we need the following basic Gysin formula of Pragacz |l8|, Prop. 2.2], whose 
proof comes from jll], cf. [|l0[ App. F]. 

Lemma 2. Let E and F be vector bundles of ranks e and f on a variety X . Let 
< d < min(e, /). Let G = Gv(d,F) be the Grassmann bundle, with projection 
tt : G — > X and universal exact sequence 0—>A^F^Q—>0. Let q = f — d, 
R = (e — d) q , and R' = (e) q . For any partitions A and fi, with A of length at most 
q, and any a € A*{X), 

n*{s R ' + \(Q - E)s IM (A - E)nir*a) = s R+x ^(F - E) na . 

We also need the following special case of the factorization formula of Lascoux 
and Schutzenberger jl(| and Berele and Reger ||, cf. p8[ . 

Lemma 3. Let E and F be vector bundles of ranks e and f . Let R — (e)-* . Let A 
be a partition of length at most f . Then 

s\(F)s R (F - E) = s R+x (F - E) . 

Note that this identity follows from Lemma ^. Finally we need the basic identity 
§1-5]: 

Lemma 4. For bundles E±, E2, and E3, and a partition /J,, 

s„(E 3 -E 1 )=Y^ ^ T s a {E 2 - £i)s r (£ 3 - E 2 ) , 

the sum over partitions a and t with \<j\ + |r| = \fi\, with c% T the Littlewood- 
Richardson coefficient. 

Now we can prove ( |3 . 1| ) . First use Lemma|4]to replace each factor s^ (Ai + i— Ai) 
that occurs in each s fl (A.) on the left side of ( |3.1| ) by the sum 

= c °Um*) (o<)*T(i) - E i) ■ 

Note that s a ^{Qi) = if £(cr(i)) > rank(Qi) = — r^. Next use Lemma || to 
replace each s a{i - ) (Q i ) ■ s R i.(Qi - £^-1) in the result by s R /. +a ^(Qi - E^i). The 
left side of (3.1) becomes 

e*(*o e Pri^uV 

|U <r(i) : T(i) \j=l / 

TT* fjj (Qi --E'i-l)Sr(i-l)(^i ~ -^i-l) n [G]^ . 



12 



ANDERS SKOVSTED BUCH AND WILLIAM FULTON 



Finally, n applications of Lemma ^ yields 

ti"* flJsR<+ ff (i)(Qi - Ei-i)s T (i-i)(Ai -£ t -i)n[G]j 

71 

= J^s_R. +0 .(j) >T (j_i)(i?i - Ei-i) n [H] , 

i=l 

and this gives the required formula 

Cn—l \ n 

II <U,-r(i) II »*+<rW,r«-l) - ^i-l) n [fl] 
i=l / z=l 

= ^c A (r)s A (£;.)n[#]. 

A 

Although we have stated it for varieties over a field, the theorem (and its proof) 
extend readily to schemes of finite type over a regular base, as in || §20]. 

3.3. A generalization. There is a generalization of the theorem which may be 
useful in its own right, and which gives some insight into the proof. (It is not 
needed in this paper.) Fix Eq, ...,E n on A, and r = (r^) satisfying ([L~2]). Let 
H = @ Hom(iJ.;_i , Ei), on which the tautological bundle maps are universal, and 
one has the universal locus f2 r C H . 

Let 7r : F — > H be the partial flag bundle parameterizing flags in each Ej of ranks 
roj,rij, . . . , Tj—ij . Let Eqj C E±j C • • • C Ej-ij c Ej denote the tautological flags 
of vector bundles on F, 1 < j < n. 

Let Z C F be the locus on which the image of -Ejj-i by the map Ej-i — ► -Ej is 
contained in the subbundle -Ey, i.e. Z is the subscheme defined by the vanishing of 
all maps -Ei,j-i — > Ej/Eij for i < j. 

One sees as in Lemma ^ that 7r maps .Z birationally onto In fact, if A is 
non-singular, this construction gives a canonical resolution of singularities of the 
universal locus tt r . It is easy to see that the class of Z is given by 

[Z\ = U*ij, 

where Zij = Cto P (Hom(Ej J _i, E i+ ij/Eij)). 

Consider a path 7 through the triangular array for r, going from roo to r nn . The 
path must be a union of line segments between neighboring rank conditions, and it 
must intersect any vertical line at most once. 

roo rn r 22 r 33 r 44 



roi r 12 r 23 r 34 

ro4 

For each j = 1, . . . , n, let kj be minimal such that 7 goes through rfe . j . Let ^(7) be 
the partial flag bundle parameterizing flags in Ej of ranks rfc.,j, r/b +i,j, . . . , Tj-ij, 
and let Efc^j C • ■ ■ C Ej-ij c Ej be the tautological flags on ^(7), 1 < j < n. If 
the path has m line segments, we let Aq, A\, . . . , A m denote the vector bundles on 
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F(-y) corresponding to the rank conditions passed through by the path. (In the illus- 
tration, m = 7, and the bundle sequence is Eqq, Eq\,Eq2, -E12, E22, E23, £34, E44.) 

Let O r (7) C F (7) be the subscheme defined by the conditions that each map 
Eij-i — > Ej/Eij vanishes for on or above the path, and rank(_E ip — > £j) < r„ 
for on or above the path and p < j. The canonical maps _F — > f 1 (7) — > X 
map Z birationally onto fi r (7) which in turn is mapped birationally onto fi r . Our 
goal is to give a formula for the class of fi r (7). To do this, we define a formal 
linear combination $(7) of symbols S(X), where A is a sequence of partitions, one 
for each line segment in 7. The formula for Q r ("f) is obtained by replacing each 
S(X(1), A(m)) by ]T™i s X ( t )(A l - and multiplying the result by \~[z i:j , 

the product over all i,j such that is on or above 7. 

We define $(7) inductively. If 7 is the lowest possible path, going from r o to 
Ton to r„„, then $(7) = S(0, ... ,0), where the empty partition is repeated 2n 
times. Otherwise, we can find a path 7' that is equal to 7 except it goes lower at 
one place, in one of the following ways: 

7' 7 



Case 1: 



Case 2: 



i— 1,_7 'i— 1, j ^*)i+l 



In Case 1, we obtain $(7) from $(7') by replacing each symbol £(..., /x, ... ) by 
T S'(. . . , £7, r, . . . ). Note that in this case we have ^(7) = F(Y) and ^(7) = 
0(7'). For Case 2, each symbol S(. . . , /U, is, . . . ) in $(7') is replaced by the symbol 
S(. . . , (Rij + v, fj,), . . .). Note that the partitions X(i) are always empty for line 
segments on the left or right edges of the triangle. 

The proof that this polynomial gives a formula for [Cl r (7)] is similar to that of our 
main theorem: one shows that \Z] pushes forward to the class of this polynomial. 
The induction step, in cither Case 1 or 2, is more transparent, as changes are made 
in only one segment of the formula. 

If bundle maps and flags of subbundles are given on X corresponding to ranks 
on or above the path 7, these determine a section s : X — > ^(7), so one has 
corresponding formulas for the classes s*[Q r (7)]. When 7 is the horizontal path 
across the top of the diagram, we recover the main theorem. 

4. On the coefficients c\(r) 

There are some properties of the coefficients c\(r) that follow from geometry, i.e. 
from the main theorem, although they are not obvious from the algorithm defining 
them. We describe these first, and then discuss properties we believe for other 
reasons. We conclude with a comparison of the numbers c\(r) with Littlewood- 
Richardson coefficients. 



Consider the dual sequence 



-> &n-l f h 



with dual rank conditions, rank(_Ej — > EY) < 77,-, which we denote by r v ; then 
f2 r v = Q r (E.). Note that the rectangular array for r v is obtained by reflecting 
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that for r in a vertical line, and replacing each rectangle by its transpose. Using 
the basic identity that s\(F — E) = s\>(E v — F v ), where A' is the transpose of A, 
we find that 

(4.1) c x .(^) = c x (r), 

where, if A = (A(l), . . . , A(n)), we put A v = (A(n)', . . . , A(l)'). 

It can happen that for some k, all of the rank conditions rank(£'i — > Ek) < Tik 
and rank(_Efc — > Ej) < rjy follow from other rank conditions. This happens when, 
in the rectangle diagram, all the rectangles on the two 45° lines descending from 
position k are empty. For the example G\ — > G2 — > G3 — > F3 — > F2 — > Fi 
considered at the end of Section |2| with rank conditions r coming from w — 3142 £ 
S4, the bundles Gi and F2 are inessential in this way. If an inessential bundle Ek is 
omitted, one has a shorter sequence E', : E — > • • • — » — > -Efe+i — ► • • • — » £J n , 
with the map -Efc-i — > -Efc+i being o and corresponding rank conditions 
r'; the array of rectangles for r' is obtained by omitting the 45° lines of empty 
rectangles and moving all rectangles below up a row. For example, if G\ and F% are 
omitted from the example, one has G2 — > G3 — ► -F3 — > Fi, with rectangular array 

□ 

□ 

□ 

Lemma ^ can be used to expand any s p (Ek+i — Sfe-i) occurring in the formula 
c n{ r ') s n{E'.) as a sum °f terms of the form s a (Ek —Ek-i) ■ s T (Ek+i — Ek). Since 
Q, r t(E' m ) = f2 r (£■.), with this interpretation we have 

(4-2) J2 C ^') S ^K) = E c^) Sx {E.) . 

Now we turn to our conjectured formula for the coefficients c\(r), which inter- 
prets them by counting Young tableaux, in a way similar to and generalizing the 
classical Littlewood-Richardson rule. Recall that a (semistandard) Young tableau 
is a filling of the boxes in the Young diagram of a partition with integers that 
are weakly increasing in rows and strictly increasing down columns. Two Young 
tableaux P and Q can be multiplied to give another Young tableau denoted P ■ Q. 
One way to do this is to arrange P and Q corner to corner and play the jeu de 
taquin, sliding inside corners through but keeping the weak and strict orderings. 

rypn prn 

For example, if P = H3 and Q = LU , this can be carried out by the sequence of 
moves 



rr 4 



The final tableau is P ■ Q. The main fact is that this product is independent 
of choice of the sequence of inside corners, from which it follows that the set of 
tableaux form an associative monoid, called the plactic monoid. With this notion, 
the Littlewood-Richardson number „ is the number of ways a given tableau T of 
shape A can be factored into a product T = P ■ Q, such that P has shape /z and Q 
has shape v. For proofs and relations with Schur polynomials, see |l5|] or @]. 

Given rank conditions r (satisfying ( |L . 2| ) as always), form the array of rectangles 
Rij . We choose a fixed tableau T,,- on each shape Rij , with the condition that each 
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entry of Tj must be strictly smaller than any entry of Tki if Rki lies in the wedge 
cut out by 45° lines below R^, i.e. if k < i and I > j with (k, I) ^ (i, j). 

From this array of rectangular tableaux we will construct a set of n-tuples of 
tableaux (Ti, . . . ,T n ) that we call factor sequences. Our conjecture is that c\(r) 
is the number of factor sequences (Ti,...,T n ) such that Ti has shape X(i) for 
1 < i < n. We first explain this for n = 3, where we start with an array of 
rectangular tableaux: 

ABC 
D E 
F 

Factor F into a product F — F x ■ F 2 of tableaux. Pass Fi up to the left, and 
multiply it to D from the right. Pass F2 up to the right and multiply it to E from 
the left. Then factor the results: 

D ■ Fx = D t • D 2 and F 2 ■ E = E x ■ E 2 . 

Pass the results up to the left and right, arriving at tableaux A ■ D\, D 2 ■ B ■ E\ 1 
and E 2 ■ C. This gives a factor sequence (Ti, T 2 , T 3 ) = (A ■ D x , D 2 ■ B ■ E u E 2 ■ C). 

In general one proceeds by induction. A factor sequence for the given array of 
rectangular tableaux is obtained by forming a factor sequence (Si, . . . , S n -i) for 
the array of the bottom n — 1 rows. Factor each Si arbitrarily into Si = Pi ■ Qi- 
Then 

(Ti, . . . , T n ) = (Tqi - Pi, ... , Qi-i ■ Ti_i^ - Pi, ... , Q n -i • In_L„) 
is a factor sequence for the given array. 

Conjecture. c\(r) is the number of factor sequences (Ti, . . . ,T n ) of shape A = 
(A(l), . . . , A(n)) that can be made from a given array of rectangular tableaux. 

The conjecture has a number of consequences: 

CI. Each c\(r) is a non-negative integer. 

C2. The coefficients c\(r) depend only on the rectangles Rij, not on their sides. 

This means that if one of the sides of a rectangle Rij is 0, the length of the other 
side is irrelevant. (The algorithm of the main theorem shows this when the height 
of a rectangle is 0, but not when the width is 0.) 

Implicit in the conjecture is the assertion 



C3. The number of factor sequences of shape A is independent of choice of fixed 
tableaux Tij . 

Granting C3, it is not hard to see that the conjectured formula for the c\(r) 



satisfies the duality (4T). For this one chooses the Tij so that no entry appears more 
than once, and uses the fact that factoring a tableau T with distinct entries into P Q 
is equivalent to factoring its conjugate T 1 into Q'-P'. It is also not difficult to verify 



that the conjectured formula satisfies the property (4.2) for omitting inessential 
bundles. 

The conjecture is true for the case where Rij is empty for j — i > 2. This follows 
from the description in Corollary [j] of Section [|, together with the fact that for a 
tableau T of rectangular shape R, for each a and r that fit together to form R, 
there is a unique factorization T = P ■ Q with P of shape a and Q of shape r; 
conversely, any factorization of T has factors of shapes that fit together to form R. 
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The conjecture has been proved when n < 3. More generally, it is proved when 
Rij is empty for j — i > 3 and no two non-empty rectangles in the third row 
are adjacent. The proof depends on a wonderful involution on pairs of tableaux 
produced for us by S. Fomin. This proof is given in 0]. 

One reason that the combinatorial formula is hard to work with is that a given 
factor sequence can arise in many ways by the algorithm that produces them. At 
first glance it would appear that to tell if some (Ti, . . . , T n ) is a factor sequence, 
one would have to test all possible ways of carrying out the sequence of factorings. 
However, there is a direct test. For this, define Pi to be the part of T^ lying to the 
right of the rectangle R i: and define Qi-i to be everything lying below T i _i ii and 




Ti = 



Then (Ti, . . . ,T„) is a factor sequence if and only if Qo and P n are empty and 
(Pi ■ Qi, ■ ■ ■ , Pn-i • Qn-l) is a factor sequence for the lower n — 1 rows of the array. 
By induction this gives a direct algorithm to test, from the top down. Note that 
this algorithm, like the theorem, uses the height of a rectangle Rij even if its width 
is zero. This criterion is proved in Q. 

The full conjecture follows from an assertion that Fomin's involution preserves 
factor sequences. This assertion is true for n < 3, and it has been verified in 500,000 
randomly generated examples for n < 10. For a discussion of Fomin's involution 
and the discussion of this, we refer again to [Q. 

The numbers c\ (r) generalize Littlewood- Richardson numbers ^ in fact as 
well as in spirit. To see this, take any rectangle R containing 7, and let r be the 
complement of 7 in R (rotated 180°): 



R = 



7 






T 



Form the array of rectangles 



R 

where the vertical lines are the height of R, the horizontal lines are its width, and 
the dot is empty. Choose r giving rise to this array. Then 

with A = (a, (3, r). This follows easily from the theorem (and also from the conjec- 
ture). 
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There are analogous conjectures for the coefficients of the polynomials for the 



more general loci described in §3.3. In particular, all these coefficients should also 



be positive. Details will be given in |4[]. 
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